The television industry is undergoing significant changes as a result of the transition from the current standard definition television (SDTV) to high definition television (HDTV). Much of this change is driven by the FCC requirement that all broadcasters in the United States must transmit all programming content as HDTV signals and must cease transmitting SDTV signals by the year 2006. As a result, high definition televisions are becoming increasingly available in the marketplace, as are HDTV conversion systems that convert an HDTV signal to an SDTV image for display on a standard definition television.
Some of the driving forces behind the transition to HDTV are the possibility of a larger and clearer picture, the changed aspect ratio (similar to movie format) in some systems, and the decreased susceptibility of the digital signal to noise during transmission to the viewer. As screens grow larger, viewers expect increased resolution. For a number of years to come, however, HDTV sets must be able to receive and display television signals according to the existing SDTV standard (e.g., PAL, NTSC, SECAM) while broadcast facilities are making the transition to the new HDTV standard (ATSC). In the interim, it is highly desirable that an HDTV set be able to display an SDTV signal at increased resolution to create the subjective impression of a high definition television image. In addition, from the broadcast side, techniques are needed which can up-convert existing standard definition (SD) materials into high definition (HD) format.
Unfortunately, the resolution of the video signal at the television receiver is limited by the quality of the original video signal (e.g., PAL, NTSC, SECAM) or the bandwidth of the transmission channel. Therefore, in order to increase the resolution of the SDTV signals for better perceptual quality, post-processing the video signal in the receiver after demodulation becomes increasingly important.
Segmentation of television images is a post-processing technique wherein each frame of an image sequence is subdivided into regions or segments. Each segment is a cluster of pixels encompassing a region of the image with a commonality of properties. For example, a segment may be distinguished by a common color, a common texture, a particular shape, an amplitude range or a temporal variation. Known early applications of segmentation include pattern recognition, target tracking, and security surveillance. Most recent research into segmentation has been in applications related to the MPEG-4 and MPEG-7 standards. In the former case, segments are identified and uniquely encoded to achieve date compression. In the case of MPEG-7, segmentation is used to identify image components for image classification and retrieval.
In the case of television image enhancement, known enhancement techniques include both global and local enhancement methods. Examples of global enhancement techniques may include the brightness and contrast controls of television (TV) receivers that control the DC offset and signal gain globally (or uniformly) over the entire image. An example of a local control enhancement technique is edge enhancement, in which an image processor automatically detects the location of edges in the image and applies appropriate enhancement only in the local region of the edge.
Although local enhancement techniques are applied only to local regions of an image, the conventional methods are nonetheless controlled by global parameters. In the case of edge enhancement, for example, the edge enhancement algorithm may adapt to the local edge characteristics. However, the parameters that govern the algorithm are global (i.e., they are the same for every region of the image). The use of global parameters places a limitation on the most effective enhancement that can be applied to any given image. A greater amount of enhancement would be available if the enhancement algorithm could be trained to recognize the features depicted in different segments of the image and could dynamically choose image enhancement parameters that are optimized for each type of image feature.
The known methods of image segmentation may be described as “hard” segmentation in that a binary decision is made. Every region either satisfies the relative criteria of a segment and is included in the desired segment, or it is completely excluded. Many conventional hard segmentation techniques are satisfactory for the applications that have been published in the prior art. However, these hard segmentation techniques are not satisfactory in many advanced applications.
For example, in the case of applying hard segmentation techniques to moving image sequences, small changes in appearance, lighting or perspective may only cause small changes is the image. The result is often that parts of the image satisfy or fail the hard segmentation criteria in a random way from image frame to image frame. When image enhancement techniques are applied exclusively to the segmented regions, the result may be random variations in the enhancement, usually at the edges of the segmented regions. Such random variations in moving sequences represent disturbing artifacts that are not acceptable to the viewers.
There is therefore a need in the art for improved apparatuses and methods for enhancing the quality of a television image. In particular, there is a need in the art for improved image enhancement techniques that are not affected by small variations in appearance, lighting, perspective, and the like between successive frames in a video image. More particularly, there is a need for improved apparatuses and methods of segmenting and enhancing a video image that do not rely on hard, binary decisions regarding whether or not to apply an enhancement technique or a segmenting technique to a pixel or group of pixels in an image.